PAC Associative Reinforcement Learning

General algorithms for the reinforcement learning problem typically learn policies in the form of a table that directly maps the states of the environment into actions. When the state-space is large these methods become impractical. One approach to increase e ciency is to restrict the class of policies by considering only policies that can be described using some xed representation. This paper pursues this approach and analyzes the associative reinforcement learning problem in the PAC learning framework. As a representation, we use a general form of decision lists that can describe a wide variety of restricted classes of policies. We then describe an algorithm that provably learns, with high probability, a good approximation of the optimal policy for any environment that satis es a particular adequacy condition stipulated for the representation used. The running time of the algorithm is polynomial in the size of the representation in addition to the usual parameters for a PAC algorithm. We give some experimental results that show that the algorithm performs well in practice. 1 Supported in part by the NSF grant CCR-9202158 and by an Andrew Mellon predoctoral fellowship at the University of Pittsburgh.